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Abstract 


The  U.S.  Navy  is  a  major  investor  in  ocean  model  development.  The  pay-off  of 
such  an  investment  is  the  value-added  ocean  nowcast/forecast  systems  on  the  naval 
operations  and  warfare  effectiveness.  The  purpose  of  this  paper  is  to  investigate  the 
value-added  of  the  Navy’s  nowcast/forecast  system  to  the  naval  antisubmarine  warfare 
(ASW)  and  anti-surface  warfare  (ASUW).  The  nowcast/forecast  versus  observational 
fields  were  used  by  the  Weapon  Acoustic  Preset  Program  (WAPP)  to  detennine  the 
suggested  presets  for  Mk  48  variant  torpedo.  The  metric  used  to  compare  the  two  sets  of 
outputs  is  the  relative  difference  in  acoustic  coverage  area  generated  by  WAPP.  Output 
presets  are  created  for  five  different  scenarios,  two  ASUW  scenarios  and  three  ASW 
scenarios  in  the  South  China  Sea.  The  same  metrics  used  in  the  nowcast/forecast  case 
were  used  to  generate  and  compare  the  acoustic  coverage.  Analysis  of  the  output  reveals 
that  the  ocean  forecast  system  outperformed  the  nowcast  system  in  most  scenarios. 
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1. 


Introduction 


Understanding  the  ocean  environment  is  imperative  and  directly  coupled  to  the 
successful  performance  of  undersea  sensors  and  subsequent  employment  of  undersea 
warfare  weapon  systems.  In  order  to  optimize  the  perfonnance  of  undersea  sensors  and 
weapons  systems,  it  is  crucial  to  gain  an  understanding  of  the  acoustic  wave  propagation 
in  the  ocean.  Having  an  accurate  depiction  of  the  ocean  environment  is  therefore  directly 
related  to  gaining  a  better  understanding  of  the  acoustic  wave  propagation. 

How  acoustic  waves  propagate  from  one  to  another  location  undersea  is 
determined  by  many  factors,  some  of  which  are  described  by  the  sound  speed  profile 
(SSP).  If  the  environmental  properties  of  temperature  and  salinity  are  known  over  the 
entire  depth,  the  SSP  can  be  calculated.  Two  approaches  are  used  to  increase  the 
accuracy  of  ocean  temperature  and  salinity  depiction:  (1)  satellite  data  assimilation,  and 
(2)  ocean  nowcast/forecast  systems.  Chu  et  al.  (2004a,  2006)  show  the  importance  of  the 
satellite  data  assimilation  for  improvement  of  the  naval  undersea  capability. 

The  U.S.  Navy  has  developed  ocean  nowcast/forecast  systems  to  determine  or  to 
predict  representative  SSP.  For  example,  Modular  Ocean  Data  Assimilation  System 
(MODAS)  is  a  commonly  used  nowcast  system,  which  is  built  on  the  base  of  the  optimal 
interpolation  (statistical  model).  The  Navy  Coastal  Ocean  Model  (NCOM)  is  a  commonly 
used  ocean  forecast  system,  which  is  built  on  the  base  of  the  Princeton  Ocean  Model 
(POM).  MODAS  uses  climatology  as  the  initial  guess  and  assimilates  satellite  and  in-situ 
measurements  such  as  altimetry,  conductivity-temperature-depth  (CTD),  expendable 
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bathythermographs  (XBT),  and  ARGO  casts.  NCOM  (physical  model)  forecasts  the 
ocean  environment  using  observational  data  such  as  temperature,  salinity,  and  velocity. 

Representation  of  the  Navy’s  nowcast  (MODAS)  and  forecast  (POM)  systems  for 
ocean  environment  (SSP  through  T,  S  profdes)  was  verified  using  the  CTD  data  collected 
from  the  South  China  Sea  Monsoon  Experiment  (SCSMEX)  in  April  -  June  1998  (Chu  et 
ah,  2001,  2004b).  The  errors  have  a  Gaussian-type  distribution  with  mean  temperature 
nearly  zero  and  mean  salinity  of  -0.2  ppt.  However,  evaluation  of  value-added  ocean 
nowcast/forecast  system  on  the  naval  undersea  capability  has  yet  been  conducted.  At  the 
combat  level,  acoustic  detection  of  torpedo  is  extremely  important  for  the  undersea 
warfare.  In  this  study,  the  Weapon  Acoustic  Preset  Program  (WAPP)  for  Mk-48  torpedo 
is  used  for  such  an  evaluation. 

2.  Oceanographic  Observations 

2.1.  South  China  Sea 

The  South  China  Sea  (SCS)  is  a  semi-enclosed  tropical  sea  located  between  the 
Asian  land  mass  to  the  north  and  west,  the  Philippine  Islands  to  the  east,  Borneo  to  the 
southeast,  and  Indonesia  to  the  south  (Figure  1),  covering  a  total  area  of  3.5  x  106  km2.  It 
includes  the  shallow  Gulf  of  Thailand  and  connections  to  the  East  China  Sea  (through 
Taiwan  Strait),  the  Pacific  Ocean  (through  Luzon  Strait),  Sulu  Sea,  Java  Sea  (through 
Gasper  and  Karimata  Straits)  and  to  the  Indian  Ocean  (through  the  Strait  of  Malacca).  All 
of  these  straits  are  shallow  except  Luzon  Strait,  the  maximum  depth  of  which  is  1800  m. 
The  complex  topography  includes  a  broad,  shallow  shelf  in  the  south/southwest;  the 
continental  shelf  of  the  Asian  landmass  in  the  north,  extending  from  the  Gulf  of  Tonkin 
to  Taiwan  Strait;  a  deep,  elliptical  shaped  basin  in  the  center,  and  numerous  reef  islands 
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and  underwater  plateaus  scattered  throughout.  The  shelf  that  extends  from  the  Gulf  of 
Tonkin  to  the  Taiwan  Strait  is  consistently  nearly  70  m  deep,  averaging  150  km  in  width; 
the  central  deep  basin  is  1900  km  along  its  major  axis  (northeast-southwest)  and 
approximately  1100  km  along  its  minor  axis,  extending  to  over  4000  m  deep.  The 
south/southwest  SCS  shelf  is  the  submerged  connection  between  southeastern  Asia, 
Malaysia,  Sumatra,  Java,  and  Borneo  and  reaches  100  m  depth  in  the  middle;  the  center 
of  the  Gulf  of  Thailand  is  about  70  m  deep. 

2.2.  South  China  Sea  Monsoon  Experiment 

SCSMEX  was  a  multi-national  large  scale  experiment  (intensive  observational 
period:  April  to  June  1998)  to  study  the  water  and  energy  cycles  of  the  Asian  monsoon 
regions  with  the  goal  (SCSMEX  Science  Working  Group,  1995)  with  shipboard 
measurements,  Autonomous  Temperature  Line  Acquisition  System  (ATLAS)  moored 
array,  and  drifters.  During  SCSMEX,  the  hydrographic  data  set  included  over  1700  CTD 
(Figure  2)  and  mooring  stations  (Chu  et  al,  2001,  2004b).  The  hydrographic  data 
collected  during  the  SCSMEX  IOP  went  through  quality  control  procedures  such  as  min- 
max  check  (e.g.,  disregarding  any  temperature  data  less  than  -2°C  and  greater  than  40°C), 
error  anomaly  check  (e.g.,  rejecting  temperature  data  deviating  more  than  7°C  from 
climatology),  ship-tracking  algorithm  (screening  out  data  with  obvious  ship  position 
errors),  max-number  limit  (limiting  a  maximum  number  of  observations  within  a 
specified  and  rarely  exceeded  space-time  window),  and  buddy  check  (rejecting 
contradicting  data).  The  climatological  data  used  for  quality  control  are  depicted  in  Chu 
et  al.  (1997a,  b).  After  the  quality  control,  the  SCSMEX  oceanographic  data  set  contains 
1742  conductivity-temperature-depth  (CTD)  and  mooring  stations.  The  majority  of  the 
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CTDs  were  nominally  capable  of  reaching  a  maximum  depth  of  2000  m.  SCSMEX 
provided  a  unique  opportunity  to  evaluate  ocean  nowcast  system  (Chu  et  ah,  2004b)  and 
forecast  system  (Chu  et  al.  2001). 

3.  Ocean  Nowcast  System 

MODAS  is  one  of  the  present  U.S.  Navy  standard  nowcast  system  for  producing 
three-dimensional  grids  of  temperature  and  salinity.  It  is  a  modular  system  for  ocean 
analysis  and  is  built  from  a  series  of  FORTRAN  programs  and  UNIX  scripts  that  can  be 
combined  to  perform  desired  tasks  (Chu  et  al.,  2004b).  MODAS  was  designed  to 
combine  observed  ocean  data  with  climatological  information  to  produce  a  quality- 
controlled,  gridded  analysis  field  as  output  (Figure  3).  The  analysis  uses  an  optimal 
interpolation  (OI)  data  assimilation  technique  to  combine  various  sources  of  data  (Fox  et 
al.,  2002). 

Chu  et  al.  (2004b)  evaluated  MODAS  using  the  SCSMEX  data.  The  errors  for 
temperature  and  salinity  nowcast  have  a  Gaussian-type  distribution  with  zero  mean  for 
temperature  and  -0.048  ppt  for  salinity  and  with  standard  deviation  (STD)  of  0.98°C  for 
temperature  and  0.22  ppt  for  salinity.  This  result  indicates  that  MODAS  usually  under¬ 
predicts  the  salinity.  The  RMSE  of  temperature  between  the  MODAS  and  SCSMEX  data 
increases  rapidly  with  depth  from  0.55°C  at  the  surface  to  1.72°C  at  62.5  m  and  then 
reduces  with  depth  to  near  0.03°C  at  3000  m  deep.  The  RMSE  of  salinity  between  the 
MODAS  and  the  SCSMEX  data  has  a  maximum  value  (0.347  ppt)  at  the  surface.  It 
decreases  to  a  very  small  value  (0.009  ppt)  at  3000  m.  Interested  readers  are  referred  to 
Chu  et  al.  (2004b). 

4.  Ocean  Forecast  System 
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NCOM,  a  popular  ocean  forecast  system,  is  based  on  two  existing  circulation 
models,  POM  and  the  Sigma/Z-level  Model  (SZM).  NCOM  has  a  free  surface  and  is 
based  on  the  primitive  equations  and  the  hydrostatic,  Boussinesq,  and  incompressible 
approximations.  Horizontal  mixing  is  provided  through  either  grid-cell  Reynolds  number 
or  the  Smagorinsky  scheme.  The  Mellor  Yamada  Level  2  or  2.5  turbulence  models 
provide  for  vertical  mixing  parameterization.  The  model  uses  an  Arakawa  C  grid,  is 
leapfrog  in  time,  and  uses  second-order  centered  spatial  finite  differences.  The 
propagation  of  surface  waves  and  vertical  diffusion  are  treated  implicitly.  NCOM  has  a 
curvilinear  horizontal  grid  and  the  vertical  grid  uses  sigma  coordinates  for  the  upper 
layers  and  z-level  (constant  depth)  coordinates  for  the  lower  layers. 

Chu  et  al.  (2001)  evaluated  POM  using  the  SCSMEX  data.  During  the  evaluation, 
the  POM  was  implemented  in  the  domain  (98.84°-l21.16°E,  3.06°S-25.07°N)  covering 
the  whole  SCS  with  horizontal  resolution  of  0.179°  by  0.175°  (approximately  20  km 
resolution)  and  vertical  resolution  of  3  sigma  levels.  The  model  uses  realistic 
bathymetric  data  from  the  Naval  Oceanographic  Office  Digital  Bathymetry  Data  Base. 

Similar  to  MOD  AS,  the  errors  of  POM  also  have  a  Gaussian- type  distribution 
with  zero  mean  for  temperature  and  -0.022  ppt  for  salinity.  The  RMSE  for  temperature  is 
0.2°C  at  the  surface,  increases  with  depth  to  a  maximum  value  of  1.2°C  (May  98)  or  of 
1.4  (June  98)  at  50-100  m  depth,  and  then  decreases  with  depth  to  a  minimum  value  of 
0.3°C  (May  98)  or  0.5°C  at  500  m.  The  mean  RMSE  is  around  0.6°C.  The  RMSE  for 
salinity  is  near  zero  at  the  surface,  increases  with  depth  to  a  maximum  value  of  0.22  ppt 
(May  98)  or  of  0.21  ppt  (June  98)  at  25-75  m  depth,  and  then  reduces  with  depth  to  a 
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minimum  value  of  0.03  ppt  at  500  m.  The  mean  RMSE  is  around  0.06  ppt.  The  interested 
readers  are  referred  to  Chu  et  al.  (2001). 

5.  Weapon  Acoustic  Preset  Program 

5.1.  General  Description 

WAPP  is  an  automated,  interactive  program  designed  to  provide  the  fleet  with  an 
onboard  means  of  generating  acoustic  presets  for  multiple  variants  of  Mk  48  torpedoes 
and  visualizing  their  performance.  Developed  by  Naval  Undersea  Warfare  Center 
(NUWC),  Division  Newport,  RI,  it  consists  of  several  elements  including  a  graphical  user 
interface  (GUI)  for  entering  various  data,  a  computational  engine  for  generating  acoustic 
performance  predictions,  and  various  forms  of  output  (Chu  et  al.,  2004a,  2006). 

The  types  of  input  data  necessary  include  tactical  (such  as  tactic  type  and  depth 
zone  of  interest),  target  (such  as  acoustic  and  Doppler  characteristics),  weapon  (such  as 
type,  mod,  and  active  or  passive  acoustic  mode),  and  environmental  information.  To 
input  the  environmental  infonnation,  the  user  selects  the  “environment”  pull-down  menu 
of  the  GUI  to  bring  up  the  Environmental  Data  Entry  (EDE)  window.  This  window, 
shown  in  Figure  4,  allows  the  entry  of  water  column  parameter  profiles  (such  as 
temperature,  salinity,  sound  speed,  and  volume  scattering  strength)  for  a  specified  latitude 
and  longitude.  Other  environmental  input  entered  via  the  EDE  consists  of  sea  surface 
conditions  (wind  speed,  wave  height,  and  sea  state)  and  bottom  conditions  (depth  and 
type).  Operationally  the  environmental  data  is  received  from  the  Sonar  Tactical  Decision 
Aid. 

5.2.  WAPP  Presetting  Process 
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Once  the  necessary  information  is  input  (or  default  values  are  selected),  WAPP  is 
ready  to  undergo  the  presetting  process.  This  process  is  begun  by  using  the  “compute” 
pull-down  menu  of  the  GUI  and  is  outlined  in  Figure  5.  The  first  step  is  to  establish  a 
valid  set  of  search  depth  (SD)  and  search  angle  (SA)  combinations.  The  program  then 
invokes  a  search  angle  selection  algorithm  to  identify  the  optimal  pitch  angle  for  each 
search  depth.  Next,  the  computational  engine  traces,  in  a  series  of  time  steps,  a  fan  of 
rays  that  bound  the  torpedo  beam  pattern  for  each  resulting  SD/SA  combination 
(Amezaga,  2006).  A  signal  excess  computation  is  performed  and  mapped  to  a  gridded 
search  region  at  each  time  step  using  the  monostatic,  active  sonar  equation  for  the 
reverberation  limited  case, 

SL  -  2TL  +  TS  -  RL  -  DT  =  SE ,  (1) 

where  SL  is  the  active  sonar  source  level,  TL  is  the  two-way  transmission  loss  between 
the  sonar  and  the  target,  TS  is  the  target  strength,  RL  is  the  reverberation  level,  and  DT  is 
the  detection  threshold.  The  signal  excess  map  is  used  to  determine  the  effectiveness 
ratio  (the  fraction  of  the  prosecutable  search  region  with  signal  excess  greater  than  0  dB, 
also  called  area  coverage)  and  laminar  distance  (the  location  of  signal  excess  center  of 
mass).  WAPP  then  ranks  the  SD/SA  combinations  based  on  these  computations  (along 
with  some  other  mitigating  factors)  and  makes  a  recommendation  as  to  the  best  preset  for 
the  given  scenario. 

In  solving  equation  (1),  the  SL,  DT,  and  TS  terms  are  based  on  properties  of  the 
sonar  system  and  target  involved,  so  they  are  selected  by  the  program  or  entered  by  the 
user,  as  is  the  case  for  TS.  The  TL  and  RL  terms  are  computed  using  a  range- 
independent,  ray  theory  propagation  model  that  accounts  for  geometric  spreading, 
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refractive  effects,  volumetric  effects,  and  boundary  interactions  with  the  ocean  surface 
and  bottom.  The  vertical  sound  speed  profdes  used  by  the  ray  tracing  model  are 
calculated  by  WAPP  from  the  temperature  and  salinity  profiles  using  the  equation 
proposed  by  Chen  and  Millero  (1977).  Geometric  spreading  and  refractive  losses  are 
determined  using  the  transmission  loss  equation  derived  using  ray  theory 


TL  =  10  log 


Rk 

8Rk  ■  a 
— -  sin  9, 

d9a 

cos# 


(2) 


V  ) 

where  Rk  is  the  horizontal  range  at  some  position  downrange,  9a  is  the  initial  angle  of  the 
ray,  and  9k  is  the  angle  of  the  ray  at  range  R,  . 

5.3.  Ranked  List-Set 

To  offer  a  means  of  user  interaction,  the  output  of  WAPP  is  in  the  form  of  a 
ranked  list-set  of  search  depths,  pitch  angles,  laminar  distances,  and  effectiveness  values. 
This  allows  the  user  to  view  all  SD/SA  combinations,  not  just  the  recommended  one,  and 
select  the  most  appropriate  one  for  the  situation.  The  list-set  is,  therefore,  a  list  of 
possible  presetting  choices  from  which  the  operator  can  choose.  In  addition,  the  ray 
traces  and  signal  excess  maps  are  viewable  using  the  GUI’s  “acoustic  coverage”  pull¬ 
down  menu.  These  forms  of  output  provide  a  visual  interpretation  of  the  acoustic 
performance  of  the  torpedo,  including  boundary  interactions  and  refraction  effects. 

Since  the  propagation  model  uses  ray  theory,  it  has  all  the  shortcomings 
associated  with  it,  such  as  being  limited  to  higher  frequencies.  In  this  case,  this  is  an 
acceptable  condition  because  the  Mk  48  torpedo  has  a  suitably  high  operating  frequency. 
Another  deficiency  of  ray  theory  is  the  poor  handling  of  shadow  zones  due  to  the 
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assumption  that  no  acoustic  energy  leaks  out  of  the  ray  tube.  This  is  also  acceptable 
because,  from  a  weapon  presetting  standpoint,  it  is  unrealistic  to  direct  a  torpedo  to  home 
in  on  a  target  in  a  shadow  zone,  so  an  accurate  description  of  the  sound  field  there  is  not 
necessary.  Finally,  ray  theory  has  the  issue  of  causing  energy  to  approach  infinity  at 
caustics  and  turning  points.  This  last  concern  is  mitigated  through  the  use  of  a  caustic 
correction  that  modifies  the  propagation  equations,  thereby  avoiding  the  case  where  the 
denominator  becomes  zero,  and  approximates  the  signal  level  near  the  caustic. 

Because  the  propagation  model  is  range-independent,  it  assumes  cylindrical 
symmetry,  meaning  it  does  not  have  range-varying  properties.  For  example,  sound  speed 
is  a  function  of  depth  only  and,  since  bathymetry  is  absent,  a  flat,  homogeneous  bottom  is 
used.  Therefore,  the  resulting  ray  traces  are  assumed  to  be  valid  for  any  direction  from 
the  source  location,  as  the  model  environment  looks  the  same  down  any  bearing  (Etter, 
1991;  Medwin  and  Clay,  1997).  This  is  not  ideal  for  determining  accurate  sound 
propagation  characteristics,  especially  in  regions  where  the  oceanography  changes 
rapidly  with  horizontal  distance,  and  could  affect  the  weapon  presets.  Under  less  variable 
conditions,  this  shortcoming  would  probably  have  little  or  no  affect  on  the  weapon 
presets,  as  the  typical  Mk  48  torpedo  engagement  would  only  involve  a  few  kilometers  of 
ocean.  Regardless,  there  is  an  effort  currently  underway  to  utilize  the  Comprehensive 
Acoustic  Sonar  Simulation  for  range-dependent  performance  predictions  for  torpedo 
presetting.  The  assumption  of  range  independence  is  consistent  with  the  large  number  of 
areas  where  there  is  little  to  no  bathymetric  variation  over  torpedo  detection  ranges  and 
also  with  cross-slope  predictions  in  more  variable  environments,  and  so  provides  a 
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reasonable  assessment  of  the  importance  of  satellite  altimetry  data  using  the  current 


weapon  system. 

6.  Relative  Difference 


The  acoustic  area  coverage  (AC)  for  different  SD/SA  combination  is  selected  as 
the  key  output  for  the  evaluation.  Since  the  SCSMEX  (T,  S)  profiles  are  treated  as 
‘ground  truth’  environmental  data,  the  WAPP  output  with  the  SCSMEX  data  is 
considered  as  the  reference  solution  for  the  weapon  system.  Absolute  relative 
differences  (RD)  in  the  key  WAPP  output  between  using  the  SCSMEX  data  and  the 
ocean  nowcast/forecast  data, 


RD 


M 


ACm-AC0 

acm 


RDP= 


ACP-AC0 

ACP 


(3) 


can  be  treated  as  the  deviation  from  the  WAPP  reference  solution.  The  less  the  RDm  (or 
RDp),  the  better  the  environmental  input  from  the  nowcast/forecast  systems  to  WAPP. 
Here,  the  subscripts  ‘M’  denotes  MODAS,  ‘P’  denotes  POM  and  ‘O’  denotes  observation 
(i.e.,  the  SCSMEX  data). 

The  presetting  process  has  generated  pairs  of  list-sets  in  which  some  SD/SA 
combinations  were  the  same  and  others  were  different.  The  list-set  can  be  thought  of  as  a 
list  of  presetting  choices;  the  choices  on  one  list  sometimes  matched  those  on  the  other 
list  and  sometimes  not.  The  instances  in  which  WAPP  produce  different  SD/SA 
combinations  for  a  profile  pair  are  the  cases  in  which  an  actual  engagement  would  have 
greater  potential  for  a  different  outcome  because,  given  these  different  choices,  the 
torpedo  would  not  be  searching  at  the  same  depth,  looking  at  the  same  search  angle,  or 
both.  Determining  the  sensitivity  of  WAPP  to  input  differences  in  these  cases  is 
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important  because  of  the  potential  for  weapon  effectiveness  to  be  affected.  The  thing  to 
remain  aware  of  here  is  that  the  actual  environment  is  whatever  it  is,  regardless  of 
differences  in  the  SCSMEX-MODAS  (or  SCSMEX-POM)  pair  profiles.  In  the  cases 
where  the  same  SD/SA  combinations  (same  choices)  are  generated  for  the  pair  profiles 
(SCSMEX-MODAS  or  SCSMEX-POM),  the  outcome  of  the  engagement  would  be  very 
similar,  subject  to  other  targeting  considerations,  because  the  same  presets  and 
environment  are  involved. 

Figure  6  depicts  two  torpedo  engagement  simulations  conducted  by  the  Naval 
Undersea  Warfare  Command  (NUWC)  -  Newport,  where  there  is  a  0.2  relative 
difference  of  acoustic  coverage  in  the  torpedo  acoustic  cone.  Each  dot  (in  red)  is  a 
probable  contact  until  the  acoustic  cone  of  the  torpedo  passes  over  the  dot.  The  dot  turns 
yellow  when  the  torpedo  has  a  detection  opportunity.  The  torpedo  then  enters  into  its 
detection,  acquisition,  and  verification  phases.  If  a  dot  remains  in  the  acoustic  cone  long 
enough  to  complete  the  detection,  acquisition,  and  verification  phases,  the  torpedo  will 
likely  enter  homing  with  a  green  dot. 

In  the  first  case  (Figure  6a),  94.2%  of  tracks  enter  the  acoustic  cone  and  46.7% 
enter  homing  with  an  overall  coverage  score  of  47.7  %.  In  the  second  case  (Figure  6b), 
when  the  acoustic  coverage  was  reduce  by  20%,  89.6%  of  tracks  enter  the  acoustic  cone 
and  only  16.3%  enter  homing  with  an  overall  coverage  score  of  33.8%.  In  other  words, 
a  relative  difference  of  0.2  leads  to  an  engagement  with  1/3  as  likely  to  the  mission 
success.  So,  a  relative  difference  of  0.2  is  large  enough  to  change  an  engagement.  A 
regression  curve  (not  shown  here)  that  is  bound  by  the  by  first  and  second  case  infers  that 
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a  RD  between  0.10  and  0.15  would  yield  an  overall  coverage  score  between  47.7%  and 
33.8%. 

7.  Sensitivity  of  WAPP  to  Ocean  Model  Uncertainty 

7.1.  Procedure  of  the  Test 

Figure  7  shows  the  procedures  of  the  evaluation.  The  SCSMEX-MODAS  and 
SCSMEX-POM  data  pairs  (temperature  and  salinity  profiles)  are  taken  as  environmental 
input  into  WAPP.  WAPP  then  calculates  the  sound  speed  from  the  respective 
temperature  and  salinity  grid  point  pairs.  The  same  default  values  for  volume  scattering 
strength  and  surface  and  bottom  conditions  were  used  for  each  run.  This  procedure  is 
repeated  for  the  SCSMEX-MODAS  (or  SCSMEX-POM)  profiles  from  April  to  June 
1998  (SCSMEX  Intensive  Observation  Period)  in  the  SCS  for  the  five  tactical  scenarios. 
The  tactical  scenarios  are  selected  using  the  Acoustic  Preset  GUI  (Figure  4).  The  five 
tactical  scenario  selected  were  high  Doppler  anti  surface  warfare  (HD  ASUW),  low 
Doppler  anti  surface  warfare  (LD  ASUW),  low  Doppler  shallow  anti  submarine  warfare 
(LD  shallow  ASW),  high  Doppler  shallow  anti  submarine  warfare  (HD  deep  ASW),  and 
low  Doppler  shallow  anti  submarine  warfare  (LD  deep  ASW).  Shallow  ASW  is  defined 
as  maximum  target  depth  of  213  m,  and  deep  ASW  is  define  as  maximum  target  depth  of 
396  m. 

The  WAPP  output  is  a  ranked  list-set  of  different  SD/SA  combination  and 
acoustic  coverage  generated  for  the  aforementioned  tactical  scenario  between  using  the 
respective  modeled  (MODAS  or  POM)  and  observed  (SCSMEX)  temperature  and 
salinity  fields.  A  configuration  management  program  which  included  a  statistical 
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software  package  was  employed  to  compare  the  generated  list  set.  Any  differences  in  the 
output  (RD)  were  attributed  to  the  ocean  model  uncertainty. 

7.2.  Statistical  Characteristics  of  RDM  and  RDP 

The  errors  in  ocean  nowcast  system  (e.g.,  MODAS)  and  forecast  system  (e.g., 
POM)  may  have  an  effect  on  the  output  of  WAPP,  depending  on  the  sensitivity  of  WAPP 
to  changes  in  input.  The  cases  highlighted  here  have  fairly  significant  differences  in  the 
temperature,  salinity,  and  sound  speed  fields. 

In  each  of  the  five  scenario  histograms  for  MODAS,  the  frequency  has  peak  at 
RDm  =  0.10  (or  0.15).  For  HD  deep  ASW,  the  frequency  of  different  SD/SA 
combinations  is  very  small  for  small  RDM  (Figure  8),  increases  with  RDM  to  150  at  RDM 
=  0.10  (162  for  RDm  =  0.15),  and  then  decreases  with  RDm  to  8  at  RDm  =  0.20  (6  at  RDm 
=  0.25).  In  other  words,  the  peak  frequency  is  at  RDM  =  0.15. 

Similarly,  in  each  of  the  five  scenario  histograms  for  POM,  the  frequency  has 
peak  at  RDP  =  0.10.  For  HD  deep  ASW,  the  frequency  of  different  SD/SA  combinations 
is  very  small  for  small  RDP  (Figure  9),  increases  with  RDP  to  282  at  RDP  =  0.10,  and  then 
decreases  with  RDP  to  25  at  RDP  =  0.15,  and  to  2  at  RDP  =  0.20.  In  other  words,  the 
peak  frequency  is  at  RDP  =  0.10. 

The  mean  RDm  and  RDP  (Figure  10)  have  the  following  features:  (a)  the  mean 
RDP  is  always  less  than  RDM  for  all  the  five  scenarios;  (b)  they  are  smaller  for  the  three 
ASW  scenarios  than  for  the  two  ASUW  scenarios;  and  (3)  RDm  is  nearly  0.2  and  RDP  is 
about  0.12  for  HD  and  LD  ASUW  scenarios.  The  highest  mean  RDm  (0.1983)  and  RDP 
(0. 1273)  are  for  the  high  Doppler  ASUW  tactics.  The  lowest  mean  RDM  is  0.0966  are  for 
the  low  Doppler  deep  ASW  tactics.  The  lowest  mean  RDP  (0.0758)  is  for  the  low 
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Doppler  deep  and  shallow  ASW  tactics.  Smaller  values  of  relative  difference  using  POM 
than  using  MODAS  (RDp  <  RDm)  in  all  five  tactics  may  imply  that  the  ocean  forecast 
system  (physical  model)  outperforms  than  the  ocean  nowcast  system  (statistical  model)  in 
the  WAPP  prediction  for  torpedo  Mk-84. 

7.3.  Probability  of  RD  over  Thresholds 

The  probabilities  of  RD  being  greater  than  0.1  and  0.15, 

//j  =  Pr  ob  (RD  >0.1),  n2  =  Pr  ob  (RD  >0.15),  (4) 

are  used  for  the  model  evaluation.  From  the  preceding  discussion  it  is  apparent  that,  in 
some  of  the  scenarios,  WAPP  output  was  quite  sensitive  to  changes  in  input 
environmental  fields,  such  as  MODAS  and  POM  versus  SCSMEX.  For  MODAS,  the  jux 
values  range  from  0.2375  (low  Doppler  Deep  and  Shallow  ASW)  to  0.81  (high  Doppler 
ASUW)  and  the  /u2  values  range  from  0.015  (low  Doppler  deep  ASW)  to  0.71  (high 

Doppler  ASUW)  (see  Table  1).  For  POM,  the  jux  values  vary  from  0.03  (low  Doppler 
Deep  and  Shallow  ASW)  to  0.55  (low  and  high  Doppler  ASUW)  and  the  /u2  values  vary 

from  0.0025  (high  Doppler  deep  ASW)  to  0.2121  (high  Doppler  ASUW)  (see  Table  2). 
This  suggests  that  the  sensitivity  of  WAPP  is  extremely  variable  and,  therefore,  so  is  the 
chance  of  affecting  the  outcome  of  an  engagement.  Based  on  this  sensitivity  analysis,  the 
ocean  nowcast/forecast  systems  contributed  50%  (POM)  to  80%  (MODAS)  chance  of 
having  a  different  engagement  outcome  in  the  ASUW  scenarios  (once  again,  assuming 
0.1-0.15  is  enough  of  a  relative  difference  in  area  coverage  to  change  the  outcome),  but 
4%  (POM)  to  40%  (MODAS)  chance  of  having  a  different  engagement  outcome  in  the 
ASW  scenarios. 
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For  each  scenario,  the  values  of  (jux,/i2)  are  greater  using  MODAS  (Figure  11) 


than  using  POM  (Figure  12).  In  each  model  (MODAS  or  POM),  the  values  of  ( //, ,  //2 ) 
are  greater  for  the  ASUW  scenarios  than  for  the  ASW  scenarios.  For  example,  jux  with 
POM  is  0.03-0.06  for  the  three  ASW  tactical  scenarios;  on  the  other  hand,  //j  with 
MODAS  is  0.24-0.44  for  the  three  ASW  tactical  scenarios. 

8.  Conclusions 

Usefulness  of  ocean  nowcast  system  (MODAS)  and  forecast  system  (POM)  for 
the  naval  undersea  warfare  is  evaluated  using  the  Weapon  Acoustic  Preset  Program  with 
observational  (T,  S)  data  collected  from  SCSMEX.  The  overall  value  of  ocean 
nowcast/forecast  systems  is  assessed  by  its  effect  on  the  outcome  of  actual  engagement, 
or  weapon  effectiveness.  The  value  could  then  be  based  on  whether  or  not  the  outcomes 
were  affected  positively,  which  in  an  ASW  (or  ASUW)  engagement  typically  means  the 
torpedo  hit  the  target  versus  missed  it.  In  this  study,  torpedo  performance  in  the  real 
world  was  not  readily  quantifiable  because,  although  the  SCSMEX  observational  data  is 
certainly  closer  to  the  actual  environmental  conditions,  there  is  no  way  to  relate  the 
weapon  (i.e.,  torpedo)  performance  predictions  to  the  expected  real  world  performance. 
The  only  real  world  performance  assertion  is  made  to  single  out  the  different  SD/SA 
combinations  for  the  sensitivity  analysis  that  the  engagement  would  have  been  very 
similar  if  the  weapon  is  assigned  the  same  presets. 

This  study  shows  that  the  full  physical  ocean  forecast  system  (POM) 
outperformed  the  statistical  nowcast  system  (MODAS)  in  all  5  tactical  scenarios.  The 
ocean  forecast  system  (POM)  had  smaller  relative  differences  in  acoustic  coverage  than 
the  ocean  nowcast  system  (MODAS)  comparing  to  the  WAPP  with  observational  T,  S 
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profiles  from  SCSMEX.  The  sensitivity  analysis  also  confirmed  that  the  probability 
decreases  with  increasing  tactic  depth  that  is  in  agreement  with  earlier  study  (Chu  et  ah, 
2006). 

The  scenarios  in  which  WAPP  is  the  most  sensitive  are  the  ones  where  the  model 
input  (MODAS  or  POM)  differed  significantly  from  observational  input  (SCSMEX), 
especially  in  the  depth  zone  of  interest  for  the  given  tactic  (ASW  or  ASUW).  The 
environmental  model  uncertainty  causes  uncertainty  in  the  SSP  characteristics,  which  in 
turn  leads  to  large  differences  in  the  sound  propagation  predictions  made  by  WAPP  for 
the  ocean  environmental  model  and  observational  fields,  and  thus  to  large  relative 
differences  in  area  coverage. 

To  quantify  the  effect  on  weapon  effectiveness,  a  two-part  study  needs  to  be 
conducted.  Part  1  would  compare  the  output  of  WAPP  using  modeled  (e.g., 
nowcast/forecast  systems)  and  in  situ  measurements  of  the  local  environment  (e.g., 
SCSMEX  here).  The  in  situ  measurements  could  be  performed  by  any  number  of  assets, 
such  as  a  U.  S.  Navy  ship  during  an  exercise  or  a  research  vessel,  although  the  area 
should  be  one  with  large  variability,  such  as  in  the  Gulf  Stream  or  Kuroshio  Current,  to 
obtain  the  most  benefit  from  the  ocean  nowcast/forecast.  Of  course,  as  with  any 
experiment  involving  in  situ  measurements,  the  data  set  will  be  much  smaller  than  the 
one  used  in  this  study. 

With  this  type  of  comparison,  any  differences  in  WAPP  output  could  be 
correlated  to  the  torpedo’s  predicted  real  world  performance  and,  therefore,  so  could  be 
the  benefit  of  any  new  model  development.  For  example,  if  the  predicted  perfonnance  is 
similar  for  the  MODAS  field  with  the  new  development  and  the  in  situ  data,  but  the 
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performance  differed  appreciably  for  the  MODAS  field  without  such  a  development,  the 
new  model  development  would  be  quite  valuable.  On  the  other  hand,  the  new  model 
development  would  be  deemed  as  being  less  beneficial.  Of  course,  the  predicted 
performance  is  still  not  real  world  perfonnance,  however. 

To  even  better  assess  the  effect  of  the  ocean  nowcast/forecast  systems  on  weapon 
effectiveness,  Part  2  would  need  to  include  simulations  of  torpedo  engagements.  The 
Weapons  Analysis  Facility  at  NUWC,  Division  Newport  has  the  capability  to  simulate 
engagements  using  torpedo  hardware-in-the-loop  and  a  high  fidelity  virtual  environment. 
Using  the  Weapon  Analysis  Facility  and  presets  generated  by  the  ocean  nowcast/forecast 
fields  and  in  situ  data  in  Part  1,  many  virtual  torpedo  engagements  could  be  conducted  to 
examine  the  effects  of  the  different  nowcast/forecast  fields  on  virtual  performance.  This 
could  be  done  for  any  number  of  scenarios,  by  alternately  using  presets  generated  by  each 
of  the  environmental  inputs  to  WAPP  and  then  comparing  the  ratios  of  hits  to  misses  for 
the  virtual  engagements. 

This  experiment  introduces  an  operational  element  by  enabling  the  presets  to  be 
chosen  by  an  operator  for  each  engagement.  It  would  also  eliminate  the  need  to  use  the 
relative  difference  in  area  coverage  and  the  associated  uncertainty  in  the  threshold  that 
produces  changes  in  engagement  outcome.  This  is  because  the  proposed  metric,  the  hit- 
miss  ratio,  is  not  a  prediction  of  performance  (like  area  coverage)  but,  rather,  a  direct 
assessment  of  it  (once  again,  in  a  virtual  environment).  Aside  from  the  cost  and  logistics 
prohibitive  alternative  of  putting  many  torpedoes  in  the  water,  an  experiment  such  as  this 
would  provide  the  next  best  analysis  of  the  value-added  ocean  nowcast/forecast  systems 
with  regard  to  torpedo  effectiveness. 
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Figure  1.  Geography  and  isobaths  showing  the  bottom  topography  of  the  South  China 
Sea.  Numbers  show  the  depth  in  meter. 
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Figure  3.  Flow  chart  of  MODAS  operational  procedure. 
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Figure  4.  Weapon  acoustic  preset  module  display. 
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Figure  5.  Flow  chart  of  WAPP  presetting  procedure. 
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Figure  6.  Horizontal  acoustic  coverage  for  a  torpedo:  (a)  with  a  typical  acoustic  cone  (left 
panel),  and  (b)  with  a  20%  reduced  acoustic  cone  (right  panel). 
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Figure  7.  Procedure  of  the  ocean  nowcast/forecast  systems  using  WAPP. 
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MODAS  vs  OBS  HD  Deep  ASW 


□  OBS  MODAS  HD  Deep  ASW 


Relative  Difference 


Figure  8.  Histogram  of  RDM  (in  %)  for  HD  deep  ASW  scenario  (mean  value:  0.113; 
standard  deviation:  0.0488). 
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POM  vs  OBS  HD  Deep  ASW 


□  OBS  POM  HD  Deep  ASW 


Relative  Difference 


Figure  9.  Histogram  of  RDP  (in  %)  for  HD  deep  ASW  scenario  (mean  value:  0.0898; 

standard  deviation:  0.0295). 
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MODAS  and  POM  Mean  RD 


RD 


Tactical  Scenario 


— MODAS  Mean  RD 
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Figure  10.  Mean  RDm  and  RDp  (in  %)  for  five  scenarios. 
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MODAS  RD  for  5  Tactial  Scenario 


Figure  11.  Two  probabilities  (//p//2)  (in  %)  of  RDM  for  the  five  tactical  scenarios. 
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POM  RD  for  5  Tactical  Scenario 


Figure  12.  Two  probabilities  (/ux,ju2)  (in  %)  of  RDP  for  the  five  tactical  scenarios. 
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Table  1.  Overall  sensitivity  of  weapon  acoustic  preset  to  MOD  AS  in  the  South  China 

Sea  simulation. 


Scenario 

Prob  (RDm>0.1) 

Prob(RDM>0.15) 

Mean  RDM 

Std  Dev  of 

rdm 

HD  Deep  ASW 

0.4375 

0.0325 

0.113 

0.0488 

LD  Deep  ASW 

0.2375 

0.015 

0.0966 

0.0441 

LD  Shallow  ASW 

0.2575 

0.03 

0.1004 

0.0476 

HD  ASUW 

0.81 

0.71 

0.1983 

0.0789 

LD  ASUW 

0.735 

0.6525 

0.1804 

0.0776 

Table  2.  Overall  sensitivity  of  weapon  acoustic  preset  to  POM  in  the  South  China  Sea 

simulation. 


Scenario 

Prob  (RDP>0.1) 

Prob(RDP>0.15) 

Mean  RDP 

Std  Dev  of 
RDP 

HD  Deep  ASW 

0.06 

0.0025 

0.0898 

0.0295 

LD  Deep  ASW 

0.03 

0.0075 

0.0759 

0.0356 

LD  Shallow  ASW 

0.0325 

0.01 

0.0758 

0.0362 

HD  ASUW 

0.54 

0.2121 

0.1273 

0.0579 

LD  ASUW 

0.55 

0.1325 

0.1208 

0.0551 
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